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THEORETICAL ACCURACY FOR 
ESTL BIT ERROR RATE TESTS 


1. Introduction 


“Bit error rate” [BER] for the purposes of this paper is the fraction of binary bits 
which are inverted by passage through a communication system. BER can be 
measured for a block of sample bits by comparing a received block with the transmitted 
block and counting the erroneous bits. Bit Error Rate [BER] tests are the most common 
type of test used by the ESTL for evaluating system-level performance. 

The resolution of the test is obvious: the measurement cannot be resolved more 
finely than 1 IN, the number of bits tested. The tolerance is not. This paper examines 
the measurement accuracy of the bit error rate test. It is intended that this information 
will be useful in analyzing data taken in the ESTL. 

This paper is divided into four sections and follows a logically ordered 
presentation, with results developed before they are evaluated. However, first-time 
readers will derive the greatest benefit from this paper by skipping the lengthy section 
devoted to analysis, and treating it as reference material. The analysis performed in this 
paper is based on a Probability Density Function [PDF] which is developed with greater 
detail in a past paper, Theoretical Accuracy for ESTL Probability of Acquisition Tests, 
EV4-98-609. 


2. Assumptions 

The entire test as well as this derivation hinge on one central assumption: it is 
valid to represent the transmitter/path/receiver system with a constant, P e , the probability 
of a bit error, which is assumed to be uniformly constant for every bit. If errors occur in 
bursts or blocks, then P e for a bit depends on whether the previous bit was an error and 
this analysis is invalid (errors must be independent). If symbols are not equally error- 
prone, then P e is a function of the data stream. If the transmitter power varies, if the 
receiver noise temperature varies (e.g., during warm-up), if the path loss varies, then P e 
is not a constant but is a function of these variables. 


3. Analysis 

The exact confidence interval is somewhat complicated and is impractical to 
calculate for large sample sizes. A very good approximation will also be developed. 


3.1 Exact Tolerance 

An exact approach was developed for Bernoulli trials (repeated independent 
instances of an experiment that can have one of two possible outcomes) in Theoretical 
Accuracy for ESTL Probability of Acquisition Tests, EV4-94-618. The result was, 
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where 


P e is the actual probability of bit error for the transmitter/attenuator/receiver 
configuration. This number is the unknown we seek to bound. 

N\s the number of samples (bits) per trial. 

n is the number of bit errors in a trial (n will be measured). 

CL is the confidence level desired for the interval (0.95 will be used here). 


P eMp is the upper boundary of the interval that contains P e with confidence CL. 
P e j 0 is the lower boundary of the interval that contains P e with confidence CL. 


and C(N,n) is the combinations function, which gives the number of combinations of n 
objects that can be chosen from a population N. 


C(N,n) = 


N\ 


n\(N-n)\ 

The integral must be solved by using the integration by parts technique 
recursively, giving a closed form of 
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This same integral reduces to a single-loop algorithm with good numerical properties in 
that it recursively adds two numbers of same sign and similar magnitude. 


1=0 . 0E+0 

for i=0 to N-n step 1 

I=(I*i + (1-x) A i) / (n+l+i) 
next i 
I=I*x A (n+l) 


Practically, the equations can be solved easily by a computer using either 
symbolic, numeric, or direct methods. It is not obvious from these equations that the 
tolerance converges independently of A^for N»n. The general solution for any n is 
complicated, so convergence will be demonstrated by solving for the relatively simple 
case of n= 1 and showing that for large N the tolerance is a constant. The rule of 

integration by parts, J udv = uv - 1 vdu , is used. 
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Now substitute from the definition for positive tolerance, t+=(P eyUp -n/N)/(n/N)=NP eyUp -l for 
n= 1, and take the limit for large N\ 
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This final equation can be solved (at least numerically) for its single unknown, t+\ if 
CL= 0.95, then r+=4.572. This illustrates the point that for N»n, t is a function of n (=1 
for this case) and of course the desired level of confidence. 

Results for this section were found numerically and are included in the next 

section. 


3.2 Exact Tolerance, Error-Free Case 


For the relatively trivial case of no errors (n=0), the closed-form solution can be 
simplified. Here the equation has been slightly modified so that P e>up is the BER we are 
CL certain that the true BER is below. 
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Results are given in the next section. 
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3.3 Gaussian Approximation of Tolerance 


For large values of N where n~N/ 2, the shape of the binomial distribution 
approaches the Gaussian “bell-curve.” For the binomial distribution the expected value 
and variance of n are 

» = NP e . a 2 =NP e (l-P e ) 

So the variance is a parabolic function of the system BER, it is zero for i^O or P e =l and 
it peaks at P e = 0.5. Combining these facts, we can find a worst case confidence interval 
in the conventional fashion. First, define a half-width parameter e such that 


p = e 

N 


P = Ye 

e,up N 


P(P lo <P e <P)>CL 


P(--e<P,<— + e)>CL 
N e N 

The Standard Normal distribution (F(z)=F((x-ju)l a) (tables available in most probability 
textbooks) can be applied to find a value for P(-a<z<a)= P(-a<(x-ju)lo<a). Then, 
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This is still a general application of the Gaussian distribution. It assumes that Pe is 
known. We will now substitute P e ~n/N to approximate variance and average. Later it 
will be shown that a better substitution would be P e ^{n+\)/N\ however, n» 1 for the 
region where the approximation of tolerance developed in this subsection is useful. 



This last equation can be solved using the Standard Normal distribution. For a 95% 
confidence level (CZ=0.95), 
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This is an expression for tolerance as a fraction of the measured error rate, 

±100e/(n/N)%. The result can be simplified further for either end of the curve: 
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The general shape of the curve is dominated by n, with a 1 / V2 improvement in 
tolerance as the BER approaches 0.5. Although this approximation is good to a couple 
of digits for more than about 400 errors, it is not good to a single digit for fewer than 43 
errors because the probability density function in this region is asymmetrically “bell” 
shaped. Results are illustrated in the next section. 


3.4 Ideal BER function of Total Received Power 


The probability of bit error for an “ideal” receiver using Binary Phase Shift Keying 
[BPSK] or Quadrature Phase Shift Keying [QPSK] is derived in most digital 
communications textbooks (see references). Many other modulation schemes have a 
similarly shaped ideal case. The general solution is, 


K = Q 


2 E u 
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where Eb is the energy in a bit, No/2 is the Power Spectral Density [PSD] of the noise at 
the receiver input, and Q(-x ) is the area under the Standard Normal Probability Density 
Function between -oo and x. k is a constant (composed of noise temperature, noise 
figure, bit rate, and Boltzman’s constant) and P T rp is the Total Received Power. So, 

P TRPl _ Q \pJ- 
P TRP 2 Q l (P e2 ) 2 

This result is used in the next section to determine how much difference in power 
exists in the tolerance interval of the Bit Error Rate. 


3.5 Expected Value of True BER 

The expected (or “average”) value of the true BER, based on a measurement, is 
ordinarily the measured BER. This is absolutely true when the probability density 
function for the true BER, given the measurement, is symmetrical about the measured 
value. The PDF is found in this paper to be approximately symmetrical (to a couple of 
digits) for measurements of 400 or more errors, and also for measured BERs near 0.5. 

For small numbers of errors, the expected true BER is actually slightly higher 
than the measurement. That is, if more measurements were made, several 
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measurements will more often than not average out to a higher BER than the initial 
measurement indicated. This effect should be accounted for in some situations. 

The “average” outcome for any probability model may be determined by 
summing the possible outcomes, each weighted by its probability or frequency of 
occurrence (and divide by the sum of the weights, which is one-- one of the possible 
outcomes has to happen). For a continuous model, integrate over the range of possible 
outcomes, the product of the outcome and the PDF evaluated at that outcome: 

oo 

E(X) = | x • f\ ( x)dx 


where 

dis a random variable. 

E(X) is the Expected Value of the random variable X. 
f(x) is the Probability Density Function of devaluated atx. 

For the binomial distribution, the PDF is given by 

m ) = (N + 1) • C(N, n ) ■ p; • (1 - P e ) (N - n) 

for P e between 0 and 1. This was developed in Theoretical Accuracy for ESTL 
Probability of Acquisition Tests, EV4-94-618, and represents the ordinary binomial 
distribution scaled by the number of possible outcomes so that it obeys a fundamental 
rule for PDFs: the area beneath it is one. Applying these definitions to the problem at 
hand, 

E(P e ) = (d + 1) • C(N, n) ■ f‘x-x" -(1-x ) {N ~ n) dx 

JO 
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where 


P e is the actual probability of bit error for the transmitter/attenuator/receiver 
configuration. This number is the unknown for which we seek an expectation. 

N is the number of samples (bits) per trial. 


n is the number of bit errors in a trial (n will be measured). 


The integral must be solved by using the integration by parts technique 
recursively, giving a closed form of 
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This same integral reduces to a single-loop algorithm with good numerical properties in 
that it recursively adds two numbers of same sign and similar magnitude. 


1=0 . 0E+0 

for i=0 to N-n step 1 

I=(I*i + (1-x) ^i) / (n+2+i) 

next i 
I=I*x^ (n+2) 


Further, a pseudo-code algorithm which gives / for the interval of 0 to 1 is, 

1=1/ (n+2) 
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for i=l to N-n step 1 
I=I*i/ (n+2+i ) 
next i 

1=1* (N+l) *C (N, n) 

Numerical evaluation of this algorithm indicates that for N»n, the expression 

E{P e )*(n + \)l N 

is at the least a very, very good approximation of expected value for the BER; however, 
the equations have proved too complicated to easily demonstrate that they truly reduce 
to this equivalence in the limit for large N. 


4. Results 


Results of applying the equations derived in the preceding section are illustrated 
in Figure 1 and tabulated in Table 1. 

The approximation developed in the previous section can be expanded for other 
common choices of confidence level. The numerator is derived from the standard 
normal distribution, provided as a table in any probability textbook: 


CL 

±e/(n/N) ~ 

0.99 

2.575835 J-- — 


V n N 

0.95 

1.959961 J-- — 


V n N 

0.9 

1.644853 J-- — 


\n N 


Having calculated the 95% confidence case, then 31% more samples are needed for 
99% confidence, and 16% fewer samples are needed for 90% confidence. 

The error-free case may be summarized for most laboratory needs as follows: 
forx>l, if a 3.00-10' bit trial is error-free, then the BER is 95% likely to be below 
1.00- 10’*. And if a 1 .00- 10 A bit trial is error-free then it is 95% certain that the BER is 
below 3.00-10"'. 
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BER Measurement Accuracy 
95% Confidence Level 
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Bits in Error (n) 


Figure 1. Tolerance of BER test for 95% Confidence Level 
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Table 1. Calculated Results for 95% Confidence Level 


Errors 

n 

True 

Positive 

Tolerance 

True 

Negative 

Tolerance 

Gaussian 

(Symmetrical) 

Approximation 

1 

+457.2% 

-76.0% 

±196.0% 

2 

+261.1% 

-69.2% 

±138.6% 

3 

+ 192.1% 

-63.8% 

±113.2% 

4 

+ 155.9% 

-59.4% 

±98.0% 

5 

+ 133.3% 

-55.9% 

±87.7% 

6 

+ 117.6% 

-53.1% 

±80.0% 

7 

+ 105.9% 

-50.6% 

±74.1% 

8 

+97.0% 

-48.5% 

±69.3% 

9 

+89.8% 

-46.7% 

±65.3% 

10 

+83.9% 

-45.1% 

±62.0% 

11 

+78.9% 

-43.6% 

±59.1% 

12 

+74.6% 

-42.3% 

±56.6% 

13 

+71.0% 

-41.1% 

±54.4% 

14 

+67.7% 

-40.0% 

±52.4% 

15 

+64.9% 

-39.0% 

±50.6% 

16 

+62.3% 

-38.1% 

±49.0% 

17 

+60.0% 

-37.2% 

±47.5% 

18 

+58.0% 

-36.5% 

±46.2% 

19 

+56.1% 

-35.7% 

±45.0% 

20 

+54.4% 

-35.0% 

±43.8% 

21 

+52.8% 

-34.3% 

±42.8% 

22 

+51.3% 

-33.7% 

±41 .8% 

23 

+50.0% 

-33.1% 

±40.9% 

24 

+48.7% 

-32.6% 

±40.0% 

25 

+47.6% 

-32.0% 

±39.2% 

26 

+46.5% 

-31.6% 

±38.4% 

27 

+45.4% 

-31.1% 

±37.7% 

28 

+44.5% 

-30.6% 

±37.0% 

29 

+43.6% 

-30.2% 

±36.4% 

30 

+42.7% 

-29.8% 

±35.8% 

31 

+41.9% 

-29.4% 

±35.2% 

32 

+41.1% 

-29.0% 

±34.6% 

33 

+40.4% 

-28.6% 

±34.1% 

34 

+39.7% 

-28.3% 

±33.6% 

35 

+39.0% 

-27.9% 

±33.1% 

36 

+38.4% 

-27.6% 

±32.7% 

37 

+37.8% 

-27.3% 

±32.2% 

38 

+37.2% 

-27.0% 

±31.8% 

39 

+36.6% 

-26.7% 

±31.4% 
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Table 1. Calculated Results for 95% Confidence Level (continued) 


Errors 

n 

True 

Positive 

Tolerance 

True 

Negative 

Tolerance 

Gaussian 

(Symmetrical) 

Approximation 

40 

+36.1% 

-26.4% 

±31 .0% 

41 

+35.6% 

-26.1% 

±30.6% 

42 

+35.1% 

-25.9% 

±30.2% 

43 

+34.6% 

-25.6% 

±29.9% 

45 

+33.7% 

-25.1% 

±29.2% 

50 

+31.8% 

-24.0% 

±27.7% 

60 

+28.6% 

-22.2% 

±25.3% 

70 

+26.3% 

-20.8% 

±23.4% 

80 

+24.4% 

-19.6% 

±21 .9% 

90 

+22.8% 

-18.6% 

±20.7% 

100 

+21.5% 

-17.7% 

±19.6% 

110 

+20.4% 

-16.9% 

±18.7% 
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Upper Bound Bit Error Rate 


Exact Upper Bound on BER Given an Error-Free (n=0) Measurement 
95% Confidence Level 



Bits Tested (N) 

Figure 2. Upper Boundary on BER for an Error-Free Trial 
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Figure 3. Power in the Tolerance Interval for the Ideal BER Curve 
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Bit Error Rate 


Tolerance Interval for the Ideal BER Curve 
95% Confidence Level 



Figure 4. Tolerance Interval Boundaries on the Ideal BER Curve 
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5. Summary and Conclusion 


BER measurement by sampling is a reasonable estimator. Tolerance of the 
measurement for small numbers of errors is an asymmetric function strongly influenced 
by the number of errors and weakly influenced by the number of samples. Asymmetry 
occurs because the distribution (PDF) for measurable BERs for small numbers of errors 
is single-tailed (no fewer than zero are possible) but for larger numbers of errors takes 
on a bell shape. At higher error rates there still may be significant variation in the 
measurements but the variation tends to be symmetrical. 

If more accuracy is required, it can be obtained by decreasing confidence or by 
increasing the number of samples taken per trial. 

A phenomenon called “kick-up” has been frequently observed in the ESTL for 
measurements of small numbers of errors. “Kick-up” describes the measurement at the 
lowest error rate, if it causes the rate of fall from left to right of the logarithmic plot of 
BER vs. RF power level to appear to diminish (the slope of this curve should 
monotonically become increasingly negative with increasing power). Kick-up causes 
concern because if it is accurate it suggests that the terminal BER at strong signal is 
something higher than error-free. Kick-up is shown to be irregular by the results 
presented here. Because the tolerance interval is “very” asymmetrical for fewer than 
about ten errors most measurements will be slightly below average, balanced by fewer 
but farther measurements on the high side. In fact, most measurements will kick down. 
Because the 95% confidence interval around a measurement will fail to contain the true 
BER for one in twenty measurements, a single kicked-up measurement does not justify 
drawing the conclusion that kick-up exists; rather, it suggests that the test should be 
repeated to gather more data and test the hypothesis that kick-up occurred. 
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